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Abstract 

In this paper, we develop interval estimation methods for means of bounded random vari- 
ables based on a sequential procedure such that the sampling is continued until the sample 
sum is no less than a prescribed threshold. 

1 Inverse Sampling 

It is a ubiquitous problem to estimate the means of bounded random variables. Specially, the 
problem of estimating the probability of an event can be formulated as the estimation of the mean 
of a Bernoulli variable. Moreover, in many applications, one needs to estimate a quantity fi which 
can be bounded in [0, 1] after proper operations of scaling and translation. A typical approach is 
to design an experiment that produces a random variable A distributed in [0, 1] with expectation 
(j,, run the experiment independently a number of times, and use the average of the outcomes as 
the estimate [2]. 

The objective of this paper is to develop interval estimation methods for means of bounded 
variables based a sequential sampling scheme described as follows. 

Let A E [0, 1] be a bounded random variable defined in a probability space (fi, Pr) with 
mean value E[A] = fj, 6 (0, 1). We wish to estimate the mean of X by using a sequence of i.i.d. 
random samples X±, X2, ■ ■• of X based on the following inverse sampling scheme: 

Continue sampling until the sample size reach a number n such that the sample sum YlT=i 
is no less than a positive number 7. 

We call this an inverse sampling scheme, since it reduces to the classical inverse binomial 
sampling scheme [3] [1] in the special case that A is a Bernoulli random variable. 
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2 Interval Estimation 



When the experiment of inverse sampling is completed, it is desirable to construct a confidence 
interval for the mean value \x in terms of the random sampling number n. For this purpose, we 
have 

Theorem 1 LetJT(z,^) = In (£) + (£ - l) In (jEl£) for < z < 1 and < /i < 1. Let 5 E (0,1). 
Let Ji be a random variable such that Ji = 1 for n < 7 + 1 and that 

\n — 1 / 7 n — 1 

/or n > 7 + 1. Ze£ fi be a random variable such that 



Then, 



\n —J 7 — n 



Pr{/i < /i < /x} > 1 - <5. 



The proof is given in Appendix A. It should be noted that, due to the monotone property 
of the function J4?(z, /x) with respect to /z, the confidence limits can be readily determined by a 
bisection search method. 

In the special case that X is a Bernoulli random variable and 7 is an integer, the confidence 
interval can be constructed in a slightly different way as follows. 

Theorem 2 Let 5 € (0, 1). Let Ji be a random variable such that Ji = 1 for n = 7 and that 

\n / 7 n 
for n > 7. Lei be a random variable such that 

\n —J 7 — n 

Then, 

Pr{/x < fi < Ji} > 1 - 5. 

The proof is given in Appendix B. As mentioned earlier, the confidence limits can be readily 
determined by a bisection search method. 

Theorems [T] and [2] are established by employing Hoeffding's inequality [5]. If we replace the 
Hoeffding's inequality by Massart's inequality (i.e., Theorem 2 at page 1271 of [6]), which is 
slightly more conservative, we can obtain via analogy arguments explicit formulas for interval 
estimation. In this regard, we have Theorem for the general inverse sampling scheme. 
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Theorem 3 Let 5 € (0, 1) and 



2 In 



■. Define 



7 + 3 



n A + n9 



2'7 



n 



1 + 97 1 



n 



7 



+ 



1 4+(n-l)0 
1 



i-^t + aA + ^(i-^t 



/or n > 7 + 1, 
/or n < 7 + 1. 



Then, 



Pr{/x < /i < /x} > 1 - 5. 
For the inverse binomial sampling scheme (with integer 7), we have 
Theorem 4 Let 5 £ (0, 1) and = — ^- T . Define 

x ' ' 2 In 4 J 



7 3 

— n 4 + n6> 

- 7 3 
M = - + 



1 



1 



27 
n 

27 



1 + M 1 



n 



+ W1 + 07 1 



Then, 



n A + n6 

Pr{^ < n < /I} > 1 - 5. 



n 



3 Conclusion 

We have established rigorous and simple interval estimation methods for means of bounded ran- 
dom variables. The construction of confidence intervals is based on inverse sampling and requires 
little computation. The nominal coverage probability of confidence intervals is always guaranteed. 



A Proof Theorem 1 

We need some preliminary results. 

The following lemma is a classical result, known as Hoeffding's inequality [5]. 

Lemma 1 Let Xi, ■ ■ ■ ,X n be i.i.d. random variables bounded in [0, 1] with common mean value 
H € (0,1). Then, Pr ^-; 1 ^ > z} < exp (nzif (z, a)) for 1 > z > fj, = EpQ]. Similarly, 

Pr I X ' < z| < exp (nzJ^{z, a)) for0<z</_i = E[X,j]. 

The following lemma has been established by Chen [1]. 

Lemma 2 J$?((l + e)/i, /i) is monotonically decreasing with respect to e 6 ^0, ^ — lV Similarly, 
r J%f((l — e)fj,,fi) is monotonically decreasing with respect to e € (0, 1). 

Lemma 3 For any 7 > and e £ (0, 1), 

Pr(n< 7 1 <exp(7^((l + £ )/i,u)). 
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Proof. Since n is an integer, we have 



Pr < n < 



1 



M(l + e) 



Pr ^ n < 



7 



At(l + e) 



Prln< 



1 



A»(l + £* 



where e* is a number depends on /i and e such that ^nj^ 
function. Clearly, e* — - 



. Here |_-J denotes the floor 



Pr < n < 



1 



1 > e > 0. For simplicity of notation, let m = a ^ £t y Since m 
Pr{n < m} = < exp(7^((l + e)/x, /z)). 



is a nonnegative integer, it can be zero or a natural number. If m = 0, then 



Otherwise if m > 1, then 



Pr < n < 



Pr{n <m} = Pr{X x + • • • + X m > 7} = Pr{X > z}, 



where X = — - and z = 3- = u(l + e*) > u. Now we shall consider three cases. 

(i) : In the case of z > 1, we have Pr{X > z} < Pr{Y%li X t > m} = < exp(<yje ((1 + e)/z,M))- 

(ii) : In the case of z = 1, we have /i = 1 _^ £t , m = 7 and 

Pr{X>z} = Pri^X,=ml =rjPr{X l = l}<rjE[X l ]= M m 



1 V / 1 

< 



l + e* 



i=i 
7 



<exp(7^((l + e)M,/x)). 



(iii): In the case of /i < z < 1, by Lemma (TJ we have 

Pr{X >z}< exp(mzJ^(z, /j)) = exp(jJ^((l + e*)/j, 

Since e* > e, it must be true that + e) < /z(l + £*) < 1 and that J^((l + £*)n,n) < 
Jt?((l + /i) as a result of LemmaO Hence, 



Pr <^ n < 



1 



Therefore, we have shown 

Pr \ r 

for all cases. 



Pr{X >z}< exp( 7 ^((l + e)fi, n)). 



<exp(7^((l + e)/i,/x)) 



(1) 



(2) 



□ 



Lemma 4 For any 7 > and s € (0, 1) 



Pr <^ n > 



7 



/i(l-e) 



< exp 7^ 



7 _ 1 



/' 
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Proof. Since n is an integer, we have 



Piin> 



1 



M(l-e) 



Pr ^ n > 



1 



/i(l-e) 



Pr <n> 



7 



M (l-e) 



1 



where [.] denotes the ceiling function. Let £ be a number such that -jjn^e) ~ ^ = m(i-C) ^ or an -^ 
/U € (0, 1). Hence, 



Pr < n > 



/i(l-e) 



with £* satisfying 
positive integer and 



7 _ 



I 



3 



Pr < n > 



Pr < n > 



1' 



m(i-C*) 



m(i-C) 



. Clearly, 1 > C* > C > 0. Let m 



yr. Then, m is a 



Pr{n > m} = Pt{Xi H h X m < 7} = Pr{X < z} 

where X = x ' and z = (1 — C*)^- Applying Lemma [TJ we have 



Pr ^ n > 



7 



Pr{X < z} < exp(mzJf (z, //)) 
exp( 7 ^((l-C*)M,M))- 



m(i - 

Note that - A*) < ^((1 - C)m> A 4 ) as a result of 1 > C* > C > and LemmaEJ Hence, 



Pr{X < z} < exp(7^f((l - C)m>A«)) = ex P 



1 



1 



/' 



□ 



Lemma 5 For any 7 > 0, Jf? ( — j 2 — r, ^ | is monotonically decreasing with respect to e e 

f 1 

V7+M' 

Proof. Let 



1 - e 



Then, 



Let 



7 



7 1 



7/i 



7(1 + £>) - /i 



and m=-. For £> > -, we have < 2 < fi, §p > and 



/' 



<9g 





1 <9z 








2 dg 


( 






I 1 " 


^1 


- M 


-. 2 




— z 


II) 




f \ 


- 1 1 


z 




— z 



rig 

1 dz 1 dz ^ f 1 — u 



In 



In 



<0, 



5 



which implies that J^f(z,/j,) is monotonically decreasing with respect to g > ^. By the relation 
between g and e, we have that g increases as e increases and that g > ^ if and only if e > 
This proves the lemma. 

The following lemma can be shown by direct computation. 
Lemma 6 

dJt?(z,n) 1 1 



which is negative for 1 > fi > z > 0, and positive for < fj, < z < 1. Moreover, 

lim J4?(z, [a) = 0, lim J^(z, fx) = — oo, lim J^(z, fx) = — oo 

for < z < 1. 

Lemma 7 For any 7 > and 5 G (0, 1), i/iere exists e* G ^^+7*' s,ucra 

7 \ H 



7 r,/X . 



Proof. Note that 



-00 < < 0, the existence of e* is established. 



Lemma 8 For any 7 > and 6 G (2/Z 7 , 1), i/iere exists e* G ^0, ^ — lj stic/i i/iai 

In I 



2 



Lemma 9 For any 7 > and 5 G (0, 1), 

n — 1 J 7 n — 1 2 



Pr<( ( —2— < u> —2— , n> 7 +ll < |. 



□ 



lim [ — ^ -,fi I = (/i, fj,) = 0, lim ( — ^ -, \i ] = —00. 

By Lemma [5l ^ — ^ — is monotonically decreasing with respect to e G ^^q^,l^. Since 



□ 



1 

Proof. Note that 

lim Jf ((1 + e)ju, ft) = 0, lim JT((1 +e)n,n) = In/*. 

e^0 e->I-l 

By Lemma [21 ,Jif((l + e)u,u) is monotonically decreasing with respect to e G ^0, ^ — l"). Since 
In// < < for 5 G (2/x 7 , 1), the existence of e* is established. 



□ 



(3 



Proof. By LemmaEJ there exists e* 6 such that Jif = By Lemma 

Pr(n>— -) <exp 1 7 J? I —1 



Therefore, to show Lemma [9j it suffices to show 

In 4 



I — ^— -,n)< — , u > -, n > 7 + U C <j n > 



n — 1 J 7 n — 1 [ u(l — e* 

Let w e /jf (^T> /") — "t 2 "' A 1 — n > 7 + l| an d n = n(o>). Let 



Note that 



which can be written as 



e = l-^. 

n/j, 



V > — ^— r > 0, 
n — 1 



1-X> " 



Hence, 



By the definition of e, we have 



and thus 



np, 7 + fx 



l>e> 



7 + V 

7 



n—1 



/x(l-e) 



Mi-£) 



i ,r y - 7 



As a result, Jf 3 ( ^ 7 r , /x I < ( ^ 7 .- ,// ) , and by Lemma[5] 

e > e . 

By this inequality and the definition of e, 

7^7 

n = > . 

u(l-e) " 

So, 



w € <! n > 



a(l-e*) 

This shows the inclusion relationship of the sets. The lemma is thus proved. □ 



Lemma 10 For any 7 > and 5 € (0, 1), 

p r Lr(2 lfl )<M, „<l)4 

\n 77 n I 2 



7 



Proof. There are two cases: Case (i) < 5 < 2/i 7 ; Case (ii) 5 > 2fjP ' . 

We first consider Case (i). Note that J$?(z,f/,) is monotonically decreasing with respect to 
z € (/U, 1) and that 

lim jriC (z, fi) = In [i. 

Since < 5 < 2/i 7 , we have ln[i > ^2-. As a result, we have that 

In - 

J?(z,n) > — 
7 

for any z E (fi, 1). This implies that |^ ( ^, //) < \i < ^ 1 is an empty set, thus the lemma 

is of course true. 

Now we consider Case (ii). By Lemma [81 there exists e* S ^0, i — 1^ such that J^((l + 
e*)fj,,(jt) — By LemmaEl 



Pr ^ n " Ml + e*) } " GXP (7 ^ ((1 + £ * )/U ' ^ = 5' 



To show Lemma [10l it suffices to show 



n / 7 n I [ n(l + e* 

Let u G | ( ^ , /i) < , < ^ 1 and n = n(w) . Let 

nfi 

Then, 

1 

< e < 1 

A* 

as a result of n > 7 > n/x and < fM < 1. By the definition of e, we have ^ = (1 + e)/x and thus 

7 

Hence, ((1 + e)/j,, ft) < + e*)/x, /i) and by Lemma 

e > e . 

By this inequality and the definition of e, 

7.7 
w = — 7^ r < 



At(l + e) ~ /i(l + e*)' 

So, 

w € <^ n < - 7 

" n(l + e*) J 
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This shows the inclusion relationship of the sets. The lemma is thus proved. □ 

Now we are in a position to prove Theorem [TJ First, we shall show that the confidence limits 
in Theorem [I] are well-defined. To this end, we need to show the existence and uniqueness of fJ,(u) 
and ~jl(u)) for any uj € For the lower confidence limit, the existence and uniqueness of /x(cj) 
follows from Lemma [6j since < < 1 for any u G Q. For the upper confidence limit, it is 
obvious that ~P(lu) = 1 for w S fi such that n(uj) < 7 + 1. For to € 0, such that n{ui) > 7 + 1, we 
have < n (^)-\ < 1> an d the existence and uniqueness of JI(uj) follows from Lemma [6j 

Second, we shall show that Pr{/x >JI} < f . By the definition of JI, we have 



7 






_lnf 


n — 


1' 


7 


7 




') 


lnf 

7 


n — 


1' 



where the second inclusion relationship can be shown as follows. 

Let u e Ijl? = ^> M>^, n > 7 + 1, // > pj and 

n = ~p = Jb(u). 

Then, n >~fi > > an d ^ (^~T ' A 4 ) — ^ (^T' A*) = as a resu lt of Lemma[6l It follows 
that 

WG JjW^f M ) <Jll /U >^_ 5n>7 + il 
\n — 1 / 7 n — 1 

and the second inclusion relationship is true. Applying Lemma [9l we have 

7 .A ^ ln l .. ^ 7 , / * 



Pr{//>/Z} <Pr^ — !— ,/i < fi> —!—, n> 7 +l \ <-. 

\n — I/7 n — 1 2 

Third, we shall show that Pr{/i < /x} < |. By the definition of /2, we have 

c {■*(!,,)< 

where the second inclusion relationship can be shown as follows. 



lnf 








7 




lnf 


,A 


3 




7 


fi< nj 



In — 

Let uj G <^ = — -, M<^, /i</x^and 



n = n(o;), /i = fJ*(uj). 
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In — 

Then, fj, < fj, < ^ and ^ /i) < /i) = — ^ as a result of Lemma[6l Hence 

toe {J^(-,fi) <^1, ^<-l 

\n J 7 n I 

and the second inclusion relationship is true. Applying Lemma [T0"1 we have 

Pr{/x<^}<Pr <^i, v<l\< 6 -. 

Finally, Theorem Q] is justified by invoking the Bonferroni's inequality. 



B Proof Theorem 2 



To show Theorem [21 we need a modified version of Lemma [Has follows. 



Lemma 11 For any 7 > and e £ (0, 1), 



Pi<n> 



7 



M(l-e) 

Proof. Since n is an integer, we have 



<exp ( 7 J^( M (l-e), M )). 



Pr < n > 



Pr < n > 







M(l-e) 



Pr ^ n > 



m(i-C* 



with satisfying -jj 



MW*) 



M(l-e) 



Clearly, 1 > (* > e > 0. Let m = nl C ) - Then, m is a 



positive integer and 

Pr{n > m} = Pr{Xi H h X m < 7} = Pr{X < z) 

where X = ^ i=1 x% and z = (1 — C*)/ 2 - Applying Lemma [TJ we have 



Pr <^ n > 



7 



M(l-e) 



Pr{X <z}< exp(mzJf(z,fJ,)) 
exp( 7 ^((l-C*)M ! M)). 



Note that ((1 - A*) < ^((1 - e)/x, /x) as a result of 1 > C* > e > and Lemma [21 Hence, 



Pr ^ n > 



7 



M(l-e) 



<exp(7^((l- e )/i,/i)). 



□ 



The remainder of the proof is similar to that of Theorem 1 and is thus omitted. 
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